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I Abstract. In this paper, we give an analogue of J0rgensen's inequality for non-elementary groups 

^ ' of isometries of quaternionic hyperbolic space generated by two elements, one of which is elliptic. As 
an application, we obtain an analogue of J0rgensen's inequality in 2-dimensional Mobius group of the 
above case. 
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1 Introduction 



J0rgensen's inequality [8] gives a necessary condition for a non-elementary two generator subgroup 
of P5L(2,C) to be discrete. Viewing PSL{2,R) as the isometry group of complex hyperbolic 1- 
space, H^, one can seek to generalize J0rgensen's inequality to higher dimensional complex hyperbolic 
00 I isometries. There has been much research in this area. Kamiya [U |T0] and Parker |14l ITS] gave 
. generalizations of J0rgensen's inequality to the two generator subgroup of PU{n, 1) when one generator 
is Heisenberg translation. By using the stable basin theorem, Basmajian and Miner [3] generalized the 
J0rgensen's inequality to the two generator subgroup of PU{2, 1) when the generators are loxodromic 
■ or boundary elliptic, and several other inequalities are due to Jiang, Kamiya and Parker [6j by using 
On . matrix method other than the purely geometric method. Jiang [7J and Kamiya [TT] gave generalizations 
of J0rgensen's inequality to the two generator subgroup of PU (2, 1) when one generator is Heisenberg 
screw motion. The generalization also was done in ^6] for the case when one generator is a regular 
^ . elliptic element. 

Following the research on complex hyperbolic space, I. Kim and J. Parker opened up the study 
of quaternionic hyperbolic space in [13]. They proved some basic facts about discreteness of two 
generator subgroups, minimal volume of cusped quaternionic manifolds, and laid some basic tools to 
study quaternionic hyperbolic space. 

It is naturally asked that theorems in complex hyperbolic space can be generalized to quaternionic 
hyperbolic space. There is an attempt in this area. Kim [12] found analogues in quaternionic hyperbolic 
space of results in [BJ [7] . 

The purpose of this paper is to show a condition for two generator subgroup of Sp{n, 1) with an 
elliptic element to be not discrete. 



*Supported by NSFs of China (No.l0801107,No.l0671004 ), NSF of Guangdong Province (No.8452902001000043) and 
Educational Commission of Guangdong Province (No. LYM08097). 

Supported by NSF of Guangdong Province (No.8152902001000004). 



1 



In order to state our theorem, we recall some facts about elliptic elements in Sp{n,l). Every 
eigenvalue of elliptic element g G Sp{n, 1) has positive or negative type [1| and its eigenvalues fall into 
n similarity classes of positive type and one similarity class of negative type. Let Aj, i = 1, - ■ ■ ,n be 
its positive classes of eigenvalues and A„+i be its negative class. Then any element in Aj has norm 1 
and the fixed point set F{g) of g in H^, contains only one fixed point if A„+i ^ Aj, i = 1, • • • , n and 
is a totally geodesic submanifold which is equivalent to or (for some m < n) if A„_|_i coincides 
with exactly m of the class Aj, i = I,-- - ,re. We call g a regular elliptic element if F(g) contains 
only one point; Otherwise g is called a boundary elliptic element. We mention here that the definition 
of regular elliptic element is slightly different from Goldman's [5] which requires its eigenvalues are 
distinct in the setting of complex number. If g £ Sp{n, 1) is elliptic, then g conjugates to 

diag{Xi, A2, • • • , A„+i), (1.1) 

where Aj G Aj, i = 1, • • • , n + 1. We define 

5(5) = max{|Ai - A„+ip : i = l,---,n}. (1.2) 

Since similarity classes Aj, i = 1, - ■ ■ , n + 1 of 5 are invariant under conjugation, we have the following 
proposition. 

Proposition 1.1. If g £ Sp{n, 1) is elliptic, then 6{g) is invariant under conjugation. 

We now deduce the formula of 6{g) defined by (1.2) for an elliptic element g G Sp{n, 1). 

We use [IT] as a reference for the properties of quaternions and matrices of quaternions, also see in 
Section 2 for an abbreviated description. Since each similarity class Aj has a unique complex number 
with nonnegative imaginary part. Let 

e'^' G Aj, i = l, - ■■ ,n + 1 
be such complex numbers, that is, < ^j < vr for each i. Let 

(Ji.n+i = max{|Ai - A„+ip : Aj G Aj, A„+i G A^+i}, i = 1, • • • , n. 

Then 

5i^n+i = maxllne'^^'u"^ - -we'^^+^w" = max{|e'^' - we'^^^+'w'^l^}. 

Let 

T{w) = e'^'w-'^e-'^^+^w + u'e'^"+^ w~^e~'^\ 

Then |e'^' - t(;e'^"+iw"^p = 2 - T{w) and 

5j,n+i = 2 - min T{w). 

\w\=l 

We take the complex representation of the quaternion w with norm 1 as 

w = wi + W2j, where wi,W2 G C. 
Then = w = wl — W2i- Note that 

•^j = }z for z G C. 
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By direct computation we have 

T{w) = 2{\wi\^C0s{ei - On+l) + \w2\^C0s{ei + On+l)). 

Hence 

n T{w) = I 

Therefore 



- 2cos(6'i + 6'n+i) if cos(6'j - ^n+i) > cos(6'i + ^n+i) 
l^iPi^^"''' \ 2cos(6'i - 6'n+i) if cos{9i - 9n+l) < cos{9i + 9n+l) 



5{g) = max{Asm'^ — — i = l,---,n}. (1-3) 

Our main theorem is 

Theorem 1.1. Let g and h be elements of Sp{n, 1). Suppose that g is an elliptic element with fixed 
point set F{g) £ ifg. // 

inf cosh^ P^^1^5{g) < 1, (1.4) 

q€F{g) 2 

then the group < g,h > generated by g and h is either elementary or not discrete. 

Applying Theorem II .11 to the subgroup PU{2, 1) of Sp{2, 1) with g being regular elliptic, we obtain 

Corollary 1.1. (cf. fT^. Theorem 3.4]) Let g and h be elements of PU {2,1) so that g is a regular 
elliptic element with unique fixed point q. If 

cosh 5{g) < 1, 

then the group {g, h) is either elementary or not discrete. 
Let 

f=('r (1.5) 

As an application, by embedding SL{2,C) to Sp{l, 1) in Section 4, we obtain 

Theorem 1.2. Let g and h be elements of SL{2,C) given by (1.5). If {g,h) is discrete and non- 
elementary. Then 

inf 4/(t)sin2 6' > 1, (1.6) 

|t|<i,iec 

where 

... _ ii - t^Riap + m + |i + ti^icp + ii - trifep + 2{i- tf 1 

/W- 4(l-|t|2)2 +2' ^ 

Choosing t = in the above theorem, we obtain 

Corollary 1.2. Let g and h be elements of SL{2,C) given by (1.5). If {g,h) is discrete and non- 
elementary. Then 

sin2e(||/i||2 + 2) > 1, (1.8) 

where = |a|2 + |6|2 + |c|2 + |d|2. 
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Remark 1.1. The J0rgensen's inequality gives that 

\tr{gf - 4| + \tr{ghg-^h-^) - 2| = Asin^ 9{1 + \bc\) > 1 

in the above case. 

Leth= (^~^^ 2^ ) • ^^^^ ^ ^"^"^ + 2) < 4(1 + \bc\) = 20, which implies that CoroUarylTM 
is better than the J0rgensen's inequality in such a case. While the case h = ^ ^ ^ having 

..._ 7 + UtM + 2tl + Wtl + 7iti + ti) , 1. , , |^,_3 

implies Theorem is weaker than the J0rgensen's inequality in such a case. 

By the above comparison of Theorem 1.2 with the J0rgensen's inequality, we show that neither 
theorem is a consequence of the other one. 



2 Preliminaries 

In this section, we give some necessary background materials of quaternionic hyperbolic geometry. 
More details can be found in [H O [13] . 

Let EI denote the division ring of real quaternions. Elements of H have the form z = zi + Z2\ + 
^sj + •24k G H where Zj G M and 

i2=f = k2 = ijk=-l. 

Let 1 = zi — Z2\ — zaj — Z4k be the conjugate of z, and 



\z\ = \fzz = ^^ zl + zl + zl + zl 

be the modulus of z. We define = (z + z)/2 to be the reaJ part of z, and Q{z) = {z — 'z)/2 to be 
the imaginary part of z. Also z~^ = z\z\~'^ is the inverse of z. Observe that ?R.{wzw~^) = 5R(z) and 
|it;zTt;~"^| = \z\ for all z and w in H. Two quaternions z and w are similar if there exists nonzero g G H 
such that z = qwq~ . The similarity class of z is the set {qzq-^ : g G M - {0}}. 

Let H"'"'^ be the vector space of dimension n+1 over EI with the unitary structure defined by the 
Hermitian form 

(z, w) =w*Jz = ttJIZi H h m^Zn - Wn+lZn+l, 

where z and w are the column vectors in V with entries (zi, - ■ ■ , Zn+i) and (wi, ■ ■ ■ , Wn+i) respectively, 
•* denotes the conjugate transpose and J is the Hermitian matrix 




We define a unitary transformation g to be an automorphism of EI"' , that is, a linear bijection such 
that {g{z), (7(w)) = (z, w) for all z and w in V. We denote the group of all unitary transformations 
by Sp{n,l). 
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Following Section 2 of [1], let 

Vb = {z G y - {0} : (z, z) = O} 
V_ = |z G y : (z, z) < o}. 



It is obvious that Vq and V- are invariant under Sp{n,\). We define to be = U Vq. Let 
P : — > C EI"' be the projection map defined by 

P{zi^ ■ ■ ■ , Zn, Zn+l) = {ziZ^^j^, ' ' ' j ZnZ^j^-^) , 

where •* denotes the transpose. 

We define = P{V^) and dH^ = P(Vo). The Bergman metric on is given by the distance 
formula 

^2 P{Z,W) (Z, W)(W, Z) ^ D-V N ^ TD-l( \ 

cosh — - — = r-. ^, where zeP (z),w £ F (w). (2.1) 

2 (z, z)(w, w) 

The holomorphic isometry group of with respect to the Bergman metric is the projective unitary 
group PSp{n, 1) and acts on P(W^'^) by matrix multiplication. 

If 5 G Sp{n, 1), by definition, g preserves the Hermitian form. Hence 

w*Jz = (z, w) = {gz, gw) = w*g*Jgz 

for all z and w in V. Letting z and w vary over a basis for V we see that J = g*Jg. From this we 
find g~^ = J^^g*J. That is: 

9 ={ * lor g = {aij)ij=i,... ^n+i = { n 

Using the identities gg~^ = g~^g = I we obtain: 

AA*-aa* = In, -Ap* + aa„+i,„+i = 0, + |an+i,n+i|^ = 1; (2.2) 

A*A-(3*(3 = In, A*a-/3*an+i,n+i = 0, -\a\^ + \an+i,n+i\^ = 1. (2.3) 

For a non-trivial element g of Sp{n, 1), we say that g is parabolic if it has exactly one fixed point 
and this lies on dH^, g is loxodromic if it has exactly two fixed points and they lie on dH-^ and g is 
elliptic if it has a fixed point in H"^. In particular, if g has fixed point go = (0, • ' ' > 0)* G H-^, then g 
has the form 

g = diag{A,a), 

where A G J7(ri;EI) and a G U{1;M). 

A subgroup G of Sp{n, 1) is called non- elementary if it contains two non-elliptic elements of infinite 
order with distinct fixed points; Otherwise G is called elementary. 

As in complex hyperbolic n-space, we have the following proposition classifying elementary sub- 
groups of Sp{n, 1). 

Proposition 2.1. (cf.J^ Lemma 2.4] (i) If G contains a parabolic element but no loxodromic ele- 
m,ent, then G is elementary if and only if it fixes a point in dH^; 

(a) If G contains a loxodromic element, then G is elementary if and only if it fixes a point in 
dH'^ or a point-pair {x, y} C dH^; 

(Hi) G is purely elliptic, i.e., each non-trivial element of G is elliptic, then G is elementary and 
fixes a point in H^. 
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3 Proof of Theorem 11.1 



Proof of Theorem 11.11 For any fixed point q £ fix{g), since cosh? ^^'^'^'^'''^ 5{g) is invariant under 
conjugation, we may assume that g is of the form (1.1) having fixed point g = (0, • • • , 0)* G ffg and 



\ P an+l,ra+l 



Then 



2p{q,h{q)) 



cosh = |a„+i,„+i| , 6{g) = max{\Xi - Xn+i\ : i = l,---,n}. 



In what fohows, we wih show that if 

\an+i,n+i\'^6{g) < 1, (3.1) 

then the group {g, h) is either elementary or not discrete. 
Let ho = h and hk+i = hkgh^^ . We write 

, _ ( _ ( A^'^ ^ 

hk - [aij )i,j=i,- ,n+i - \ ^(fc) jk) 



^n+l,n+l 



Then 



^(fe+i) ^(fc+i) \ 
g(k+l) 



where L = diag{Xi, A2, ••• , An). 
Therefore 



<:tU = «Si,n+iAn+iaS,,„+i - P^'^LipO'^r (3.2) 



and 



l^n+l,n+ll^ — (^n+l,n+lA"+l"n+l,n+l /^'''^''-^(/^'■'^^ )* ) ('^n+l,n+l A"+l'^i+l,n+l f3^^^ L* {(3^^^)*) 

= |aSi^„+J^ + /3(^)L(/3W)*/3('=)L*(/3('=))* (3.3) 



-4']i,n+iAn4-iaSi,„+i/3('^)^*(/3('=^)* - /3('=)^(/3(^'))*aSi,.+i^^ 
If there exists some k such that /J^'^) = 0, then by (2.2) and (2.3) we have 

l4+i,n+il = l and a('^)=0, 

which imphes that g is a fixed point of hk = ^fc-i^^^j'^x. We deduce that g is a fixed point of hk-i and 
|an+i,n+i| = 1 by induction. Thus g is a fixed point of /i, which imphes that {g,h) is elementary. 
In what follows, we may assume that 13^''^ / 0. 
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We first consider the case when all the elements of /3^''^ are nonzero, that is, 

/ 0,i = I,-- - ,n. 

In this case, noting that 



we have 



(fc+l) |2 ^ I (fc) |4, |^(fc)|4 (k) , (k) (S^J'') T-„('=) 1 

^ l"n+l,n+ll 1/^ I "n+l.n+l^"+l"w+l,n+lV/ . "n+li^^"n+l.J 

j=l 



l^n+l,n+l 



i=l 

Let 



= « = 1, • • • , n + 1. 

Then G A^, i = 1, - ■ ■ ,n+ 1 and 

UjU^ + -Un+iiZJ =2-\Ui- -Un+lP- 

We can rewrite (3.4) as 

n 

< l«Sl,n+ir + l/?^'^^ - E l«Sl,n+ll>Sl,/(2 - - «n+lP). (3-5) 

1=1 

Noting that -[/J^'^^p + = 1, by (3.5) we have 

n 

I (fc+1) |2 _ 1 <^ I |2 I |2| _ |2 

l'^n+l,n+ll — l<^n+l,n+ll 2^\'^n+l,i\ I'"* • 

Therefore 

- 1 < (|a?i.n+iP - l)l«Si,n+iP<^(5)- (3.6) 

We remark that the case /3^'^^ ^ with some a^^^ ^ = for t G {1, • • • , n} also leads to the above 
inequality. 

Noting (3.1), we obtain by induction 



and 



•^n+l.n+ll < l"n+l,n+ll 



|aif+;i+iP - 1 < (|a„+i,„+ip - l)(|a„+i,„+ip,5(5))'=+^ (3.7) 
Thus I 

'^n+ijU+il — ^ ^'^d {/ifc} is a sequence with distinct elements. By (2.2) and (2.3), we have 

and 



By passing to its subsequence, we may assume 
Thus hk+i converges to 

which imphes that {h,g) is not discrete. This concludes the proof. 



4 Proof of Theorem 11.21 

Proof of Theorem 11.21 As in [Ull], we can regard ^^(l, 1) as the isometrics of hyperbohc 4-space 
H'^, whose model is the unit ball in the quaternions H. S'L(2,C), the isometrics of hyperbolic 3-space 
, can be embedded as a subgroup of ^^(l, 1) as following: 

/G5L(2,C)-^T/r-iG5p(l,l), (4.1) 

where 



V2 V -j 1 

Let g and h be stated as (1.5) and g = TgT^^, h = ThT^^. Then 

In fact, g,h Sp{l, 1) can be verified directly by lemma 1.1 in Applying the formula (1.3) to g in 
which 9i = 02 = 9, we have 

6{g) = Asin^e. 

It is easy to know that the fixed point set of g is {t] : t £ Cwith |t| < 1}. 
Let z = ^ ^ and w = hz. Then 

and 

1/ \|2 .V 1 M/^ |t|2 + l + t + t (1 + t-t- |t|2)j \ / a c \ /(t - l)j 

=i((l-*)j'-l-*)( e d)[{-l-t + i+m \t? + l-i-t )[b d)[-l-t 

Since (z, z) = (w, w) = |tp — 1, by direct computation, we have 

1 2 p{ti,Hij)) (z, w)(w, z) 

cosh = = f{t), 

2 (z, z)(w, w) 

where f{t) be stated as in (1.7). 

Applying Theorem 11.11 to g,h €z Sp{l, 1) given by (4.3), we conclude the proof of Theorem 1.2. 
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